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Status of this Memo 


This memo provides information for the Internet community. It does 
not specify an Internet standard of any kind. Distribution of this 
memo is unlimited. 


Copyright Notice 


Copyright (C) The Internet Society (2003). All Rights Reserved. 


Abstract 


This document describes Linux Netlink, which is used in Linux both as 
an intra-kernel messaging system as well as between kernel and user 
Space. The focus of this document is to describe Netlink's 
functionality as a protocol between a Forwarding Engine Component 
(FEC) and a Control Plane Component (CPC), the two components that 
define an IP service. As a result of this focus, this document 
ignores other uses of Netlink, including its use as a intra-kernel 
messaging system, as an inter-process communication scheme (IPC), or 
as a configuration tool for other non-networking or non-IP network 
services (such as decnet, etc.). 


This document is intended as informational in the context of prior 
art for the ForCES IETF working group. 
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1. Introduction 


The concept of IP Service control-forwarding separation was first 
introduced in the early 1990s by the BSD 4.4 routing sockets [9]. 

The focus at that time was a simple IP(v4) forwarding service and how 
the CPC, either via a command line configuration tool or a dynamic 
route daemon, could control forwarding tables for that IPv4 
forwarding service. 


The IP world has evolved considerably since those days. Linux 
Netlink, when observed from a service provisioning and management 
point of view, takes routing sockets one step further by breaking the 
barrier of focus around IPv4 forwarding. Since the Linux 2.1 kernel, 
Netlink has been providing the IP service abstraction to a few 
services other than the classical RFC 1812 IPv4 forwarding. 
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The motivation for this document is not to list every possible 
service for which Netlink is applied. In fact, we leave out a lot of 
services (multicast routing, tunneling, policy routing, etc). Neither 
is this document intended to be a tutorial on Netlink. The idea is 
to explain the overall Netlink view with a special focus on the 
mandatory building blocks within the ForCES charter (i.e., IPv4 and 
QoS). This document also serves to capture prior art to many 
mechanisms that are useful within the context of ForCES. The text is 
limited to a subset of what is available in kernel 2.4.6, the newest 
kernel when this document was first written. It is also limited to 
IPv4 functionality. 


We first give some concept definitions and then describe how Netlink 
fits in. 


1.1. Definitions 


A Control Plane (CP) is an execution environment that may have 
several sub-components, which we refer to as CPCs. Each CPC provides 
control for a different IP service being executed by a Forwarding 
Engine (FE) component. This relationship means that there might be 
several CPCs on a physical CP, if it is controlling several IP 
services. In essence, the cohesion between a CP component and an FE 
component is the service abstraction. 


1.1.1. Control Plane Components (CPCs) 


Control Plane Components encompass signalling protocols, with 
diversity ranging from dynamic routing protocols, such as OSPF [5], 
to tag distribution protocols, such as CR-LDP [7]. Classical 
management protocols and activities also fall under this category. 
These include SNMP [6], COPS [4], and proprietary CLI/GUI 
configuration mechanisms. The purpose of the control plane is to 
provide an execution environment for the above-mentioned activities 
with the ultimate goal being to configure and manage the second 
Network Element (NE) component: the FE. The result of the 
configuration defines the way that packets traversing the FE are 
treated. 


1.1.2. Forwarding Engine Components (FECs) 


The FE is the entity of the NE that incoming packets (from the 
network into the NE) first encounter. 


The FE's service-specific component massages the packet to provide it 
with a treatment to achieve an IP service, as defined by the Control 
Plane Components for that IP service. Different services will 

utilize different FECs. Service modules may be chained to achieve a 
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more complex service (refer to the Linux FE model, described later). 
When built for providing a specific service, the FE service component 
will adhere to a forwarding model. 


1.1.2.1. Linux IP Forwarding Engine Model 


4--------------- t 
+->-| FW |---» | TCP, UDP, : 
+----+ 4--------------- + 
R v 
4----«----4 | Fw | 
| +----+ 
| Y 
To host From host 
stack stack 
Ingress S. Y 
device 4£------- + +|---|--+ +-------- + Egress 
-»----- >| rw |-->|Ingress|-->---->| Forw- |->| FW |->| Egress | device 
+----+ | TC | | ard | +----+ | TC |--> 
4------- t 4------- t 4-------- t 
The figure above shows the Linux FE model per device. The only 
mandatory part of the datapath is the Forwarding module, which is RFC 
1812 conformant. The different Firewall (FW), Ingress Traffic 


Control, and Egress Traffic Control building blocks are not mandatory 
in the datapath and may even be used to bypass the RFC 1812 module. 
These modules are shown as simple blocks in the datapath but, in 
fact, could be multiple cascaded, independent submodules within the 
indicated blocks. More information can be found at [10] and [11]. 


Packets arriving at the ingress device first pass through a firewall 
module. Packets may be dropped, munged, etc., by the firewall 
module. The incoming packet, depending on set policy, may then be 
passed via an Ingress Traffic Control module. Metering and policing 
activities are contained within the Ingress TC module. Packets may 
be dropped, depending on metering results and policing policies, at 
this module. Next, the packet is subjected to the only non-optional 
module, the RFC 1812-conformant Forwarding module. The packet may be 
dropped if it is nonconformant (to the many RFCs complementing 1812 
and 1122). This module is a juncture point at which packets destined 
to the forwarding NE may be sent up to the host stack. 
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Packets that are not for the NE may further traverse a policy routing 
submodule (within the forwarding module), if so provisioned. Another 


firewall module is walked next. The firewall module can drop or 
munge/transform packets, depending on the configured sub-modules 
encountered and their policies. If all goes well, the Egress TC 


module is accessed next. 


The Egress TC may drop packets for policing, scheduling, congestion 
control, or rate control reasons.  Egress queues exist at this point 
and any of the drops or delays may happen before or after the packet 
is queued. All is dependent on configured module algorithms and 
policies. 


1.1.3. IP Services 


An IP service is the treatment of an IP packet within the NE. This 
treatment is provided by a combination of both the CPC and the FEC. 


The time span of the service is from the moment when the packet 
arrives at the NE to the moment that it departs. In essence, an IP 
service in this context is a Per-Hop Behavior. CP components running 
on NEs define the end-to-end path control for a service by running 
control/signaling protocol/management-applications. These 
distributed CPCs unify the end-to-end view of the IP service. As 
noted above, these CP components then define the behavior of the FE 
(and therefore the NE) for a described packet. 


A simple example of an IP service is the classical IPv4 Forwarding. 
In this case, control components, such as routing protocols (OSPF, 
RIP, etc.) and proprietary CLI/GUI configurations, modify the FE's 
forwarding tables in order to offer the simple service of forwarding 
packets to the next hop. Traditionally, NEs offering this simple 
Service are known as routers. 
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In the diagram below, we show a simple FE<->CP setup to provide an 
example of the classical IPv4 service with an extension to do some 
basic QoS egress scheduling and illustrate how the setup fits in this 
described model. 


Control Plane (CP) 


| MA ok f OSTEN | 
COPS -\ 

ospfd | | PEP | \ | 

/ Nr ow | | 

| 

| 

| 


N 
/------ N / | j 


| | \ | | 
| | 


KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK 


Forwarding KKKKKKKKKKKKK Netlink layer KKKKKKKKKKEKK 
Engine (FE) KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKK 


IPv4 forwarding | | 


| | 
| FE Service / / | 
| Component / / 
Rete eaten OR: precoce no] A TTS 
| | | / | | 
packet | Lo =------- E ==- |----- | packet 
in | | | IPv4 | | Egress | | out 
--—»---»|------ »|----»|Forwarding|----- >| QoS |--->| ---->|-> 
| | Scheduler | | 
| | 


The above diagram illustrates ospfd, an OSPF protocol control daemon, 
and a COPS Policy Enforcement Point (PEP) as distinct CPCs. The IPv4 
FE component includes the IPv4 Forwarding service module as well as 
the Egress Scheduling service module. Another service might add a 
policy forwarder between the IPv4 forwarder and the QoS egress 
scheduler. A simpler classical service would have constituted only 
the IPv4 forwarder. 


Over the years, it has become important to add additional services to 
routers to meet emerging requirements. More complex services 
extending classical forwarding have been added and standardized. 
These newer services might go beyond the layer 3 contents of the 
packet header. However, the name "router", although a misnomer, is 
still used to describe these NEs. Services (which may look beyond 
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the classical L3 service headers) include firewalling, QoS in 
Diffserv and RSVP, NAT, policy based routing, etc. Newer control 
protocols or management activities are introduced with these new 
services. 


One extreme definition of a IP service is something for which a 
service provider would be able to charge. 


2. Netlink Architecture 
Control of IP service components is defined by using templates. 


The FEC and CPC participate to deliver the IP service by 
communicating using these templates. The FEC might continuously get 
updates from the Control Plane Component on how to operate the 
service (e.g., for v4 forwarding or for route additions or 
deletions). 


The interaction between the FEC and the CPC, in the Netlink context, 
defines a protocol. Netlink provides mechanisms for the CPC 
(residing in user space) and the FEC (residing in kernel space) to 
have their own protocol definition -- kernel space and user space 
just mean different protection domains. Therefore, a wire protocol 
is needed to communicate. The wire protocol is normally provided by 
some privileged service that is able to copy between multiple 
protection domains. We will refer to this service as the Netlink 
service. The Netlink service can also be encapsulated in a different 
transport layer, if the CPC executes on a different node than the 
FEC. The FEC and CPC, using Netlink mechanisms, may choose to define 
a reliable protocol between each other. By default, however, Netlink 
provides an unreliable communication. 


Note that the FEC and CPC can both live in the same memory protection 
domain and use the connect() system call to create a path to the peer 
and talk to each other. We will not discuss this mechanism further 
other than to say that it is available. Throughout this document, we 
will refer interchangeably to the FEC to mean kernel space and the 
CPC to mean user space. This denomination is not meant, however, to 
restrict the two components to these protection domains or to the 
same compute node. 


Note: Netlink allows participation in IP services by both service 
components. 
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2.1. Netlink Logical Model 
In the diagram below we show a simple FEC<->CPC logical relationship. 


We use the IPv4 forwarding FEC (NETLINK_ROUTE, which is discussed 
further below) as an example. 


G [POLES 

| | / CPC-2 \ 
| | CPC-1 | | cops | | 
| [esea| | Pe | | 
| | / rs o | 
| | 
| 


EAE E A ICICI ICI ICICI ICICI ICICI ICICI ICICI ICICI ICICI ICI I IEA I a | 
KKKKKKKKKKKKK BROADCAST WIRE KKKKKKKKKKKK 


| || | [Sched | 


FE---------- XCkCk kk Ck k ck kc k Ck k ck ck k ck k k ck kc k ck ck k ck kc kck k ck kc kc kc k k ck ck kk kk | 
| IPv4 forwarding | | | | 
| FEC | | | 
— — d pnus | 
|| E il | 
| | |Ingress| | IPv4 | |Egress | | 

| | police Forward | | Qos | | 
| | | | 
_ uM 


Netlink logically models FECs and CPCs in the form of nodes 
interconnected to each other via a broadcast wire. 


The wire is specific to a service. The example above shows the 
broadcast wire belonging to the extended IPv4 forwarding service. 


Nodes (CPCs or FECs as illustrated above) connect to the wire and 
register to receive specific messages.  CPCs may connect to multiple 
wires if it helps them to control the service better. All nodes 
(CPCs and FECs) dump packets on the broadcast wire. Packets can be 
discarded by the wire if they are malformed or not specifically 
formatted for the wire.  Dropped packets are not seen by any of the 
nodes. The Netlink service may signal an error to the sender if it 
detects a malformatted Netlink packet. 
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Packets sent on the wire can be broadcast, multicast, or unicast. 
FECS or CPCs register for specific messages of interest for 
processing or just monitoring purposes. 


Appendices 1 and 2 have a high level overview of this interaction. 
2.2. Message Format 


There are three levels to a Netlink message: The general Netlink 
message header, the IP service specific template, and the IP service 
specific data. 


0 1 2 3 
Od, 2: 39,4 5.6-'L:8.9:0.1 2.3.,4.5.6 71.8 9:0 12 3 475-6 7.:8^9-0:1 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


Netlink message header 
luec E EEE EE i tales E EON ATES 
| IP Service Template | 
Wa ete th Sa hag e D aL A A dA hee ene 
| IP Service specific data in TLVs 
NM ere eet Se er LACER CEU RR 
The Netlink message is used to communicate between the FEC and CPC 
for parameterization of the FECs, asynchronous event notification of 


FEC events to the CPCs, and statistics querying/gathering (typically 
by a CPC). 


The Netlink message header is generic for all services, whereas the 
IP Service Template header is specific to a service. Each IP Service 
then carries parameterization data (CPC->FEC direction) or response 
(FEC->CPC direction). These parameterizations are in TLV (Type- 
Length-Value) format and are unique to the service. 


The different parts of the netlink message are discussed in the 
following sections. 


2.3. Protocol Model 


This section expands on how Netlink provides the mechanism for 
service-oriented FEC and CPC interaction. 
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2.3.1. Service Addressing 
Access is provided by first connecting to the service on the FE. The 
connection is achieved by making a socket() system call to the 


PF NETLINK domain. Each FEC is identified by a protocol number. One 
may open either SOCK RAW or SOCK DGRAM type sockets, although Netlink 
does not distinguish between the two. The socket connection provides 
the basis for the FE<->CP addressing. 


Connecting to a service is followed (at any point during the life of 
the connection) by either issuing a service-specific command (from 
the CPC to the FEC, mostly for configuration purposes), issuing a 
statistics-collection command, or subscribing/unsubscribing to 
Service events. Closing the socket terminates the transaction. 
Refer to Appendices 1 and 2 for examples. 


2.3.2. Netlink Message Header 


Netlink messages consist of a byte stream with one or multiple 
Netlink headers and an associated payload. If the payload is too big 
to fit into a single message it, can be split over multiple Netlink 
messages, collectively called a multipart message. For multipart 
messages, the first and all following headers have the NLM F MULTI 
Netlink header flag set, except for the last header which has the 
Netlink header type NLMSG DONE. 


The Netlink message header is shown below. 


1 2 3 
12345678901234567839012345678901 
——-—4-—R-——-4--—R--4-4-4--—-4-94--—-4-94-—--—-4-4--—-4-4- 

Length | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
Type | Flags | 


—€—————^——^A——^AR—AA——^—AA—^AA^AAR^AAR^—A^AA^A4———^—^—^—^—^€^A^R^A^A^A——€A——€———^—^—^—————— 
Sequence Number | 
—fof-4-4-4-4-4-4- 4-4-4 4-4-4 pp pip pip ip ip ip ip igi gigi gig it 
Process ID (PID) | 
———^—^AR^A——«—«4—^A^A^A^A^AA^A^R^A€—€—€4————— A————— git 


qox ede 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
+ 
l 
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The fields in the header are: 


Length: 32 bits 
The length of the message in bytes, including the header. 


Type: 16 bits 

This field describes the message content. 

It can be one of the standard message types: 
NLMSG_NOOP Message is ignored. 

NLMSG_ERROR The message signals an error and the payload 
contains a nlmsgerr structure. This can be looked 
at as a NACK and typically it is from FEC to CPC. 


NLMSG_DONE Message terminates a multipart message. 


Individual IP services specify more message types, e.g., 
NETLINK_ROUTE service specifies several types, such as RTM_NEWLINK, 
RTM DELLINK, RTM GETLINK, RTM NEWADDR, RTM DELADDR, RTM NEWROUTE, 
RTM DELROUTE, etc. 


Flags: 16 bits 

The standard flag bits used in Netlink are 

NLM F REQUEST Must be set on all request messages (typically 
from user space to kernel space) 


NLM F MULTI Indicates the message is part of a multipart 
message terminated by NLMSG DONE 
NLM F ACK Request for an acknowledgment on success. 


Typical direction of request is from user 
Space (CPC) to kernel space (FEC). 

NLM F ECHO Echo this request. Typical direction of 
request is from user space (CPC) to kernel 
space (FEC). 


Additional flag bits for GET requests on config information in 


the FEC. 
NLM_F_ROOT Return the complete table instead of a 
single entry. 
NLM_F_MATCH Return all entries matching criteria passed in 


message content. 

NLM_F_ATOMIC Return an atomic snapshot of the table being 
referenced. This may require special 
privileges because it has the potential to 
interrupt service in the FE for a longer time. 


Convenience macros for flag bits: 
NLM_F_DUMP This is NLM_F_ROOT or’ed with NLM_F_MATCH 
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Additional flag bits for NEW requests 
NLM_F_REPLACE Replace existing matching config object with 
this request. 


NLM_F_EXCL Don’t replace the config object if it already 
exists. 

NLM_F_CREATE Create config object if it doesn’t already 
exist. 

NLM_F_APPEND Add to the end of the object list. 


For those familiar with BSDish use of such operations in route 
sockets, the equivalent translations are: 


- BSD ADD operation equates to NLM F CREATE or-ed 
with NLM F EXCL 

- BSD CHANGE operation equates to NLM F REPLACE 

- BSD Check operation equates to NLM F EXCL 

- BSD APPEND equivalent is actually mapped to 
NLM F CREATE 


Sequence Number: 32 bits 
The sequence number of the message. 


Process ID (PID): 32 bits 

The PID of the process sending the message. The PID is used by the 
kernel to multiplex to the correct sockets. A PID of zero is used 
when sending messages to user space from the kernel. 


2.3.2.1. Mechanisms for Creating Protocols 


One could create a reliable protocol between an FEC and a CPC by 
using the combination of sequence numbers, ACKs, and retransmit 

timers. Both sequence numbers and ACKs are provided by Netlink; 
timers are provided by Linux. 


One could create a heartbeat protocol between the FEC and CPC by 
using the ECHO flags and the NLMSG NOOP message. 
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2.3.2.2. The ACK Netlink Message 


This message is actually used to denote both an ACK and a NACK. 
Typically, the direction is from FEC to CPC (in response to an ACK 


request message). However, the CPC should be able to send ACKs back 
to FEC when requested. The semantics for this are IP service 
specific. 

0 1 2 3 


0 4.:2:.3 4: 556 T 8 Oe 0. (1:2. 3" Ae S 6 7.8 9:0) L:2.3 4:5.6 78.9 0 1 
V—4—R—-—-—4—R——L———-—4—-—4-—*-4--—.-*.-—.-—L-—— LM-L—---4--4- 
| Netlink message header | 
| type = NLMSG ERROR | 
V—4—-—-—-—4—R——L———-—R—-4-—*-—4-—.-—.-*.-—4-4-—.-— —-——-—-4-.-4- 
| Error code | 
V—4—R—4—R—4-—L——L———-—L—-4-—*-—4-—.-—.-*.-—4-—L-—— LM-——---4-.-- 
| OLD Netlink message header 
V—4———R——L——L————R—-—4-—*-—4--—.-*.-—-L-—.-— —-——---4--4- 


Error code: integer (typically 32 bits) 


An error code of zero indicates that the message is an ACK response. 
An ACK response message contains the original Netlink message header, 
which can be used to compare against (sent sequence numbers, etc). 


A non-zero error code message is equivalent to a Negative ACK (NACK). 
In such a situation, the Netlink data that was sent down to the 
kernel is returned appended to the original Netlink message header. 
An error code printable via the perror() is also set (not in the 
message header, rather in the executing environment state variable). 


2.3.3. FE System Services’ Templates 


These are services that are offered by the system for general use by 
other services. They include the ability to configure, gather 
statistics and listen to changes in shared resources. IP address 
management, link events, etc. fit here. We create this section for 
these services for logical separation, despite the fact that they are 
accessed via the NETLINK ROUTE FEC. The reason that they exist 
within NETLINK ROUTE is due to historical cruft: the BSD 4.4 Route 
Sockets implemented them as part of the IPv4 forwarding sockets. 
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2.3.3.1. Network Interface Service Module 


This service provides the ability to create, remove, or get 
information about a specific network interface. The network 
interface can be either physical or virtual and is network protocol 
independent (e.g., an x.25 interface can be defined via this 
message). The Interface service message template is shown below. 


0 1 2 3 
0 eb, <2) 3? Ae Srp T Be OF Ob 11:273 A 5. 6-778 9.:0 1, 2 3 45.6 2B. 19> 0 1 
E —— 
Family | Reserved | Device Type 
—R-—4—.—R-—R-4-.—-—L-4-.-—-—L-4-.-—-L-4-.--L-4-.--L-4---4-4--—V 
Interface Index | 
—R-—4—4—R-—R-4-.—-—L-4-.-—-—L-4-.-—-—L-4-.--L-4-.--L-4---4-4-4-V 
Device Flags | 
—R-—4——R-—R-4-.—-—L-4-.-—-—L-4-.-—-—L-4-.-—-—L-4-.--—L-4-.--—4-4--—4 
Change Mask | 
—R-—4—.—R-—R-4-—.—-—L-4-.-—-—L-4-.-—-L-4-.--—L-4-.--L-4---—-4--—V 


+—+—4+—4+—+ 


Family: 8 bits 
This is always set to AF_UNSPEC. 


Device Type: 16 bits 

This defines the type of the link. The link could be Ethernet, a 
tunnel, etc. We are interested only in IPv4, although the link type 
is L3 protocol-independent. 


Interface Index: 32 bits 
Uniquely identifies interface. 


Device Flags: 32 bits 


IFF UP Interface is administratively up. 
IFF BROADCAST Valid broadcast address set. 

IFF DEBUG Internal debugging flag. 

IFF LOOPBACK Interface is a loopback interface. 
IFF POINTOPOINT Interface is a point-to-point link. 
IFF RUNNING Interface is operationally up. 

IFF NOARP No ARP protocol needed for this interface. 
IFF PROMISC Interface is in promiscuous mode. 
IFF NOTRAILERS Avoid use of trailers. 

IFF ALLMULTI Receive all multicast packets. 

IFF MASTER Master of a load balancing bundle. 
IFF SLAVE Slave of a load balancing bundle. 
IFF MULTICAST Supports multicast. 
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IFF_PORTSEL 


IFF_AUTOMEDIA 


IFF_DYNAMIC 


Change Mask: 


July 2003 


Is able to select media type via ifmap. 
Auto media selection active. 
Interface was dynamically created. 


32 bits 
Reserved for future use. Must be set to OxFFFFFFFF. 


Applicable attributes: 
Attribute 


IFLA UNSPEC 


HHH 


I 


I 


FLA ODISC 


FLA STATS 


FLA IFNAME 
FLA MTU 
FLA LINK 


IFLA ADDRESS 
IFLA BROADCAST 


Description 


Unspecified. 

Hardware address interface L2 address. 
Hardware address L2 broadcast 
address. 

ASCII string device name. 

MTU of the device. 

ifindex of link to which this device 
is bound. 

ASCII string defining egress root 
queuing discipline. 

Interface statistics. 


Netlink message types specific to this service: 
RTM NEWLINK, RTM DELLINK, and RTM GETLINK 


2.3.3.2. IP Address Service Module 


This service provides the ability to add, remove, or receive 
information about an IP address associated with an interface. The 
address provisioning service message template is shown below. 


-+-+-+-+-+- 


Family: 


dcc Oe 
+ 
l 
+ 
l 
+ 
l 
1 
+ 
l 


6 

T— 
ly 

T— 

T— 
8 bits 


[o9] 


+ 0 


1 2 3 
Ody 2) 3 A Or ABE OOP L238! A567 BT O 0. 1 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
Length | Flags | Scope 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


Interface Index | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


Address Family: AF_INET for IPv4; and AF_INET6 for IPV6. 


Length: 


8 bits 
The length of the address mask. 


Flags: 8 bits 
IFA_F_SECONDARY For secondary address (alias interface). 


Salim, et. 


al. 


Informational [Page 15] 


RFC 3549 Linux Netlink as an IP Services Protocol 


IFA_F_PERMANENT For a permanent address set by the user. 
When this is not set, it means the address 
was dynamically created (e.g., by stateless 
autoconfiguration). 

IFA_F_DEPRECATED Defines deprecated (IPV4) address. 

IFA_F_TENTATIVE Defines tentative (IPV4) address (duplicate 
address detection is still in progress). 


Scope: 8 bits 


The address scope in which the address stays valid. 
SCOPE_UNIVERSE: Global scope. 
SCOPE_SITE (IPv6 only): Only valid within this site. 
SCOPE_LINK: Valid only on this device. 
SCOPE_HOST: Valid only on this host. 


le attributes: 


Attribute 

IFA_UNSPEC 
IFA ADDRESS 
IFA LOCAL 

IFA LABEL 

IFA BROADCAST 
IFA ANYCAST 
IFA CACHEINFO 


Description 

Unspecified. 

Raw protocol address of interface. 
Raw protocol local address. 

ASCII string name of the interface. 
Raw protocol broadcast address. 

Raw protocol anycast address. 

Cache address information. 


Netlink messages specific to this service: RTM NEWADDR, 
RTM_DELADDR, and RTM_GETADDR. 


3. Currently Defined Netlink IP Services 


Although there are many other IP services defined that are using 


Netlink, as mentioned earlier, 


we will talk only about a handful of 


those integrated into kernel version 2.4.6. These are: 


NETLINK_ROUTE, NETLINK_FIREWALL, and NETLINK_ARPD. 


3.1. IP Service NETLINK_ROUTE 


This service allows CPCs to modify the IPv4 routing table in the 


Forwarding Engine. 


It can also be used by CPCs to receive routing 


updates, as well as to collect statistics. 


3.1.1. Network Route Service Module 


This service provides the ability to create, remove or receive 
information about a network route. The service message template is 


shown below. 
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0 1 2 3 

Q Eb:2-3.4:5-.677 849 00> 1, 2-3. 4S: 6.7:8.9 Os 1 23 45.67 8 9-0. 1 
V—R—--ERTR RTT RTT BR RM BBMÁEBMBÁERE- 
| Family | Src length | Dest length | TOS | 
V—R---T--RTR-RMRTR-MRT TET BM BMB—MÁEBMÁBRÁEREM 
| Table ID | | Protocol | Scope | Type 
V—R—--ERTR--RTR-RMRTR-RMT MTM BRL BBRBE- RR 
| Flags | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+ +H 


Family: 8 bits 
Address Family: AF_INET for IPv4; and AF_INET6 for IPV6. 


Src length: 8 bits 
Prefix length of source IP address. 


Dest length: 8 bits 
Prefix length of destination IP address. 


TOS: 8 bits 

The 8-bit TOS (should be deprecated to make room for DSCP). 
Table ID: 8 bits 

Table identifier. Up to 255 route tables are supported. 


RT_TABLE_UNSPEC An unspecified routing table. 
RT_TABLE DEFAULT The default table. 

RT_TABLE MAIN The main table. 

RT_TABLE LOCAL The local table. 


The user may assign arbitrary values between 
RT TABLE UNSPEC(0) and RT_TABLE_DEFAULT (253). 


Protocol: 8 bits 
Identifies what/who added the route. 


Protocol Route origin. 

RTPROT UNSPEC Unknown. 

RTPROT REDIRECT By an ICMP redirect. 
RTPROT KERNEL By the kernel. 
RTPROT_BOOT During bootup. 
RTPROT_STATIC By the administrator. 


Values larger than RTPROT_STATIC(4) are not interpreted by the 
kernel, they are just for user information. They may be used to 
tag the source of a routing information or to distinguish between 
multiple routing daemons. See <linux/rtnetlink.h> for the 
routing daemon identifiers that are already assigned. 


Salim, et. al. Informational [Page 17] 


RFC 3549 


Scope: 8 bits 

Route scope 
RT_SCOPE_UNIVERSE 
RT_SCOPE_SITE 


RT_SCOPE_LINK 
RT_SCOPE_HOST 
RT_SCOPE_NOWHERE 


The values between RT_SCOPE_UNIVERSE (0) 


are available to the user. 


Type: 8 bits 
The type of route. 


Route type 


RTN_UNSPEC 
RTN UNICAST 
RTN LOCAL 

RTN BROADCAST 


RTN ANYCAST 

RTN MULTICAST 
RTN BLACKHOLE 
RTN UNREACHABLE 


RTN PROHIBIT 


RTN THROW 


RTN NAT 


RTN XRESOLVE 
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(valid distance to destination). 


Global route. 

Interior route in the 

local autonomous system. 
Route on this link. 

Route on the local host. 
Destination does not exist. 


and RT SCOPE SITE(200) 


Description 

Unknown route. 

A gateway or direct route. 

A local interface route. 

A local broadcast route 

(sent as a broadcast). 

An anycast route. 

A multicast route. 

A silent packet dropping route. 

An unreachable destination. 
Packets dropped and host 
unreachable ICMPs are sent to the 
originator. 

A packet rejection route. Packets 
are dropped and communication 
prohibited ICMPs are sent to the 
originator. 

When used with policy routing, 
continue routing lookup in another 
table. Under normal routing, 
packets are dropped and net 
unreachable ICMPs are sent to the 
originator. 

A network address translation 
rule. 

Refer to an external resolver 
implemented). 


(not 
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Flags: 32 bits 
Further qualify the route. 


RTM F NOTIFY If the route changes, notify the 
user. 
RTM_F_CLONED Route is cloned from another route. 


RTM_F_EQUALIZE Allow randomization of next hop 
path in multi-path routing 
(currently not implemented). 


Attributes applicable to this service: 


Attribute Description 

RTA_UNSPEC Ignored. 

RTA_DST Protocol address for route 
destination address. 

RTA_SRC Protocol address for route source 
address. 

RTA IIF Input interface index. 

RTA OIF Output interface index. 

RTA GATEWAY Protocol address for the gateway of 
the route 

RTA PRIORITY Priority of route. 

RTA PREFSRC Preferred source address in cases 


where more than one source address 
could be used. 
RTA METRICS Route metrics attributed to route 
and associated protocols (e.g., 
RTT, initial TCP window, etc.). 
RTA MULTIPATH Multipath route next hop's 


attributes. 

RTA PROTOINFO Firewall based policy routing 
attribute. 

RTA FLOW Route realm. 


RTA CACHEINFO Cached route information. 


Additional Netlink message types applicable to this service: 
RTM NEWROUTE, RTM DELROUTE, and RTM GETROUTE 
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3.1.2. Neighbor Setup Service Module 


This service provides the ability to add, remove, or receive 
information about a neighbor table entry (e.g., an ARP entry or an 
IPv4 neighbor solicitation, etc.). The service message template is 
shown below. 


1 2 3 
Lo20304 75/56. 8.90 0 2 34 5-6- T:58:9-:)09 -1 2304-5. 6- 7 8970.1 
—————————— — — — GR 
Family | Reservedl | Reserved2 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


Interface Index | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + 
State | Flags Type | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


+—+—+—+oo 


Family: 8 bits 
Address Family: AF_INET for IPv4; and AF_INET6 for IPV6. 


Interface Index: 32 bits 
The unique interface index. 


State: 16 bits 
A bitmask of the following states: 


NUD_INCOMPLETE Still attempting to resolve. 

NUD_REACHABLE A confirmed working cache entry 

NUD_STALE an expired cache entry. 

NUD_DELAY Neighbor no longer reachable. 
Traffic sent, waiting for 
confirmation. 

NUD_PROBE A cache entry that is currently 
being re-solicited. 

NUD_FAILED An invalid cache entry. 

NUD_NOARP A device which does not do neighbor 
discovery (ARP). 

NUD_PERMANENT A static entry. 

Flags: 8 bits 
NTF_PROXY A proxy ARP entry. 
NTF_ROUTER An IPv6 router. 
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Attributes applicable to this service: 


Attributes Description 

NDA_UNSPEC Unknown type. 

NDA_DST A neighbour cache network. 
layer destination address 

NDA_LLADDR A neighbor cache link layer 
address. 


NDA CACHEINFO Cache statistics. 


Additional Netlink message types applicable to this service: 
RTM NEWNEIGH, RTM DELNEIGH, and RTM GETNEIGH. 


3.1.3. Traffic Control Service 


This service provides the ability to provision, query or listen to 
events under the auspices of traffic control. These include queuing 
disciplines, (schedulers and queue treatment algorithms -- e.g., 
priority-based scheduler or the RED algorithm) and classifiers. 
Linux Traffic Control Service is very flexible and allows for 
hierarchical cascading of the different blocks for traffic resource 


sharing. 
++ ++ +----- + 4+------- + ++ ++ .++ 
e [bom po J-l oaise || — [| | 
|| | |---->|Filter|--->|Class| ———---- s ejes T ltl 
IMEEM 4| || 
| a a tas Ic nan ce mes atom ccm + : 
Qn see IB 
| | || +->|Filter|-_ +----- 00 o 4------- + ++ | M PN 
[om oem mL J] egise 1] | -> 
ei [cīass|  +------- Mam 
-»dev-»|| [| [| +------ + _->| 4----------2----- +| EE 
f-»|Filter|- | 4---------------------- + . 
4R------ * . 
|| pfoeueicue qn md M R pee nS *| || 
| | | Parent Queuing discipline hibs] 
I] dhs p oe eee omnes ene ee sae ea + || 
|| || 
E P E E EE EE * 
Parent Queuing discipline 
| (attached to egress device) 
SSS SS ee eo es + 


The above diagram shows an example of the Egress TC block. We try to 
be very brief here. For more information, please refer to [11]. A 
packet first goes through a filter that is used to identify a class 
to which the packet may belong. A class is essentially a terminal 
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queuing discipline and has a queue associated with it. The queue may 
be subject to a simple algorithm, like FIFO, or a more complex one, 
like RED or a token bucket. The outermost queuing discipline, which 
is referred to as the parent is typically associated with a 
scheduler. Within this scheduler hierarchy, however, may be other 
scheduling algorithms, making the Linux Egress TC very flexible. 


The service message template that makes this possible is shown below. 
This template is used in both the ingress and the egress queuing 
disciplines (refer to the egress traffic control model in the FE 
model section). Each of the specific components of the model has 
unique attributes that describe it best. The common attributes are 
described below. 


1 2 3 
12,94 5.6. ?.8.9:(0. 12.3 4-5 6. 7,..8 9. 0 ld 2:34 56.7.8 -9:0 t 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- +++ 
Family | Reserved1 | Reserved2 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ++ 


Interface Index | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
Qdisc handle | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
Parent Qdisc | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
TCM Info | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


+—+—4+—4+—+—+00 


Family: 8 bits 
Address Family: AF_INET for IPv4; and AF_INET6 for IPV6. 


Interface Index: 32 bits 
The unique interface index. 


Odisc handle: 32 bits 

Unique identifier for instance of queuing discipline. Typically, 
this is split into major:minor of 16 bits each. The major number 
would also be the major number of the parent of this instance. 


Parent Qdisc: 32 bits 

Used in hierarchical layering of queuing disciplines. If this value 
and the Qdisc handle are the same and equal to TC H ROOT, then the 
defined qdisc is the top most layer known as the root qdisc. 


TCM Info: 32 bits 

Set by the FE to 1 typically, except when the Qdisc instance is in 
use, in which case it is set to imply a reference count. From the 
CPC towards the direction of the FEC, this is typically set to 0 
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except when used in the context of filters. In that case, this 32- 
bit field is split into a 16-bit priority field and 16-bit protocol 
field. The protocol is defined in kernel source 
<include/linux/if_ether.h>, however, the most commonly used one is 
ETH_P_IP (the IP protocol). 


The priority is used for conflict resolution when filters intersect 
in their expressions. 


Generic attributes applicable to this service: 


Attribute Description 

TCA_KIND Canonical name of FE component. 

TCA_STATS Generic usage statistics of FEC 

TCA_RATE rate estimator being attached to 


FEC. Takes snapshots of stats to 
compute rate. 
TCA_XSTATS Specific statistics of FEC. 
TCA_OPTIONS Nested FEC-specific attributes. 


Appendix 3 has an example of configuring an FE component for a FIFO 
Qdisc. 


Additional Netlink message types applicable to this service: 

RTM NEWODISC, RTM DELQDISC, RTM_GETQDISC, RTM_NEWTCLASS, 

RTM DELTCLASS, RTM GETTCLASS, RTM NEWTFILTER, RTM DELTFILTER, and 
RTM GETTFILTER. 


3.2. IP Service NETLINK FIREWALL 


This service allows CPCs to receive, manipulate, and re-inject 
packets via the IPv4 firewall service modules in the FE. A firewall 
rule is first inserted to activate packet redirection. The CPC 
informs the FEC whether it would like to receive just the metadata on 
the packet or the actual data and, if the metadata is desired, what 
is the maximum data length to be redirected. The redirected packets 
are still stored in the FEC, waiting a verdict from the CPC. The 
verdict could constitute a simple accept or drop decision of the 
packet, in which case the verdict is imposed on the packet still 
Sitting on the FEC. The verdict may also include a modified packet 
to be sent on as a replacement. 
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Two types of messages exist that can be sent from CPC to FEC. These 
are: Mode messages and Verdict messages. Mode messages are sent 
immediately to the FEC to describe what the CPC would like to 
receive. Verdict messages are sent to the FEC after a decision has 
been made on the fate of a received packet. The formats are 
described below. 


The mode message is described first. 


0 1 2 3 
0.1:2 3 4 5 6 7.8 9 012.34 5.6. 7. 8:9 Q0 M284 5-6 7 8 9.0 1 
*—R—4—4-—4-4-t-t-t-----*-t-t-t-—t-—L-L-4— —4——4-4-4-4-4-4-4-4 
| Mode | Reservedl | Reserved2 
*—R—R—4-—4-4-t-t-t----t-t-t-t-t-—t-—L-t-4- 4 —4—4—4-4-4-4-4-4-4 
| Range | 
Fatt ata tata tartar t arta t arta t at tbat tata —4—4-4-4-4-4-4-4-4 


Mode: 8 bits 
Control information on the packet to be sent to the CPC. The 
different types are: 


IPQ_COPY_META Copy only packet metadata to CPC. 
IPQ COPY PACKET Copy packet metadata and packet payloads 
£6: CRC. 


Range: 32 bits 
If IPQ COPY PACKET, this defines the maximum length to copy. 
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A packet and associated metadata received from user space looks 
as follows. 


2 3 4 5 6 
+-+-+-+-+ 


l 
+ he 
l 


Packet ID 

a a a SS Si Se SS Se ee ee ee MH 

Mark | 

Sn a Sn Se Si Si SS Ss 

timestamp_m | 

—t-—*-t-t-—t-—t-—tL—L———4—4-—4-4-t-t-t-t-t-t-t-t-t-t-t-t-t-t-t-— 
timestamp u 

—t-—t-t-t-—t—t—t—L———4—4-—4-—4-t-t-t-t-t-t-t-t-t-t-t-t-t-t-t-— 

hook | 

Sa a Se Sa Se See SS Se ne ee 

indev_name | 

SS a Sn Se Si Si SS a ee ee ee ee ee HMH 

outdev_name | 

a an a Sa Sa Se Si SS ee ne ee ee ee H 

hw_protocol | hw_type 
=+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SS Se sn ne ee 
hw addrlen | Reserved 

Sn Sn SS Si SS SS en ee 

hw_addr | 

=+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ H+H 

data_len | 

a Se Si Si Sie Si Se Se a ee ee ee ee ee ee 

Payload... | 

Sn a SS Se Si SS Se HMH 


+— + — + — + — + — + HH HH HH HOO 


Packet ID: 32 bits 
The unique packet identifier as passed to the CPC by the FEC. 


Mark: 32 bits 
The internal metadata value set to describe the rule in which 
the packet was picked. 


timestamp_m: 32 bits 
Packet arrival time (seconds) 


timestamp_u: 32 bits 
Packet arrival time (useconds in addition to the seconds in 


timestamp_m) 


hook: 32 bits 
The firewall module from which the packet was picked. 
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indev_name: 128 bits 
ASCII name of incoming interface. 


outdev_name: 128 bits 
ASCII name of outgoing interface. 


hw_protocol: 16 bits 
Hardware protocol, in network order. 


hw_type: 16 bits 
Hardware type. 


hw addrlen: 8 bits 
Hardware address length. 


hw_addr: 64 bits 
Hardware address. 


data_len: 32 bits 
Length of packet data. 


Payload: size defined by data_len 
The payload of the packet received. 


The Verdict message format is as follows 
1 2 
O 1 6 7 

—4+-+-4+-+4+-+-4+-+4+-+-4+-4+-4- +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


w 


p 
N 
w 
A 
ol 
Oy 
- 
co 
Ne} 
co 
Ne} 
e 
= 
N 
w 
A 
ol 
OY 
-1 
co 
Ne} 
e 
= 


-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +H 
Packet ID | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +H 
Data Length | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +H 
Payload... | 
—R-—-——-—--—-—t-4--—-—L--— —L— tata — LL. l4. 


+—+—+—+—+00 


Value: 32 bits 


This is the verdict to be imposed on the packet still sitting 
in the FEC. Verdicts could be: 


NF ACCEPT Accept the packet and let it continue its 


traversal. 
NF DROP Drop the packet. 
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Packet ID: 32 bits 
The packet identifier as passed to the CPC by the FEC. 


Data Length: 32 bits 
The data length of the modified packet (in bytes). If you don’t 
modify the packet just set it to 0. 


Payload: 
Size as defined by the Data Length field. 


3.3. IP Service NETLINK_ARPD 
This service is used by CPCs for managing the neighbor table in the 
FE. The message format used between the FEC and CPC is described in 


the section on the Neighbor Setup Service Module. 


The CPC service is expected to participate in neighbor solicitation 
protocol(s). 


A neighbor message of type RTM_NEWNEIGH is sent towards the CPC by 
the FE to inform the CPC of changes that might have happened on that 


neighbor’s entry (e.g., a neighbor being perceived as unreachable). 


RTM_GETNEIGH is used to solicit the CPC for information on a specific 


neighbor. 
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5. Security Considerations 

Netlink lives in a trusted environment of a single host separated by 

kernel and user space. Linux capabilities ensure that only someone 

with CAP NET ADMIN capability (typically, the root user) is allowed 
to open sockets. 
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Appendix 1: Sample Service Hierarchy 


In the diagram below we show a simple IP service, foo, and the 
interaction it has between CP and FE components for the service 


(labels 1-3). 
The diagram is also used to demonstrate CP<->FE addressing. In this 
section, we illustrate only the addressing semantics. In Appendix 2, 


the diagram is referenced again to define the protocol interaction 
between service foo's CPC and FEC (labels 4-10). 


| CP protocol | 

. | component | 

C d | | For | 
| IP service | 

| foo | 


M . 
| 
Vv 
V 
| 
A 
| 


FOR [kei eese ICI ICICI ICI IE eiie 
KkKKKAKAKKKE Netlink layer ****** KKK KH 
* 


ACkk ck ck ck kk kkk|*kkk*k | KKKKKKKKKK | KKKKKKKKKK 


> 


FE component/module 
for IP Service 
--->---|------ >--- | foo |----- »----- 


mds 


The control plane protocol for IP service foo does the following to 
connect to its FE counterpart. The steps below are also numbered 
above in the diagram. 


1) Connect to the IP service foo through a socket connect. A typical 


connection would be via a call to: socket(AF NETLINK, SOCK RAW, 
NETLINK FOO). 
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2) Bind to listen to specific asynchronous events for service foo. 
3) Bind to listen to specific asynchronous FE events. 
Appendix 2: Sample Protocol for the Foo IP Service 


Our example IP service foo is used again to demonstrate how one can 
deploy a simple IP service control using Netlink. 


These steps are continued from Appendix 1 (hence the numbering). 
4) Query for current config of FE component. 


5) Receive response to (4) via channel on (3). 


6) Query for current state of IP service foo. 
7) Receive response to (6) via channel on (2). 


8) Register the protocol-specific packets you would like the FE to 
forward to you. 


9) Send service-specific foo commands and receive responses for them, 
if needed. 


Appendix 2a: Interacting with Other IP services 


The diagram in Appendix 1 shows another control component configuring 
the same service. In this case, it is a proprietary Command Line 
Interface. The CLI may or may not be using the Netlink protocol to 
communicate to the foo component. If the CLI issues commands that 
will affect the policy of the FEC for service foo then, then the foo 
CPC is notified. It could then make algorithmic decisions based on 
this input. For example, if an FE allowed another service to delete 
policies installed by a different service and a policy that foo 
installed was deleted by service bar, there might be a need to 
propagate this to all the peers of service foo. 
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Appendix 3: Examples 


In this example, we show a simple configuration Netlink message sent 
from a TC CPC to an egress TC FIFO queue. This queue algorithm is 
based on packet counting and drops packets when the limit exceeds 100 
packets. We assume that the queue is in a hierarchical setup with a 
parent 100:0 and a classid of 100:1 and that it is to be installed on 
a device with an ifindex of 4. 


l 
+ he 
l 


-+-+-+-+-+-+-+-+-+-+-+-+- 
lags (NLM_F_EXCL | 
|NLM_F_CREATE | NLM_F_REQUEST) 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
Sequence Number (arbitrary number) | 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
Process ID (0) | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 


-+-+-+-+-+-+-+-+-+-+-+-+-+-+- 
Type (RTM_NEWQDISC) 


l 
— + 
l 


1 
+ 
| 
+ 
| 
| 
+ 


—+—+——+—+00 


|Family (AF_INET)| Reserved1 | Reservedl 
€—————R—————5à——————EA—A—A——————^^—A—^ AA———————— 
| Interface Index (4) | 


+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| Qdisc handle (0x1000001) | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| Parent Qdisc (0x1000000) | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| TCM Info (0) | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| Type (TCA_KIND) | Length (4) 

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| Value ("pfifo") | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
| Type (TCA OPTIONS) | Length (4) 

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-++ 
| Value (limit=100) | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 
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